1. Field of the Invention
The present invention relates to an image processing device, and particularly to an image processing device for performing demosaicing processing of a mosaic image, and to a processing method thereof and a program for causing a computer to execute the same.
2. Description of the Related Art
Solid-state imaging devices such as CCD Charge Coupled Device) image sensors and CMOS (Complementary Metal Oxide Semiconductor) image sensors are normally configured with photoreceptors arrayed in a grid, with electric charges occurring due to photoelectric conversion at each photoreceptor being sequentially read out. Since a normal photoreceptor has a singular spectral property, the color of image signals acquired from a solid-state imaging device is that of one channel (i.e., monochromatic). To obtain color images (e.g., tri-channel images such as RGB) with a single solid-state imaging device, a solid-state imaging device is employed which has filters with differing spectral properties (colors) for each photoreceptor. An imaging apparatus using one such color solid-state imaging device is often called a single-sensor apparatus or single-chip apparatus. Since a single-channel image is obtained from the color solid-state imaging device, each pixel only acquires color of the filter of the corresponding photoreceptor, and consequently, an image which is mosaic-like image with regard to color is obtained. Due to this reason, an output image of such a color solid-state imaging device is called a mosaic image.
In order to obtain a multi-channel image from a color imaging device, there is the need to interpolate between the color information of each pixel of the mosaic image according to the surrounding pixel positions, using appropriate image processing. Such image processing is generally called demosaicing processing, color interpolation processing, synchronization, and so forth.
As can be understood from the above description, demosaicing processing is indispensable for single-sensor color imaging apparatuses using a color solid-state imaging device, and various techniques have been developed over the years.
An issue in demosaicing processing is that the sampling frequency and phase of each color of the color filter is different, affecting the image quality in many ways. With the primary-color Bayer array (hereafter, referred to simply as “Bayer array”) which is the most commonly-used color array today, color filters of the three primaries of R (Red), G (Green), and B (Blue) are used, with G being arrayed in a checkerboard pattern and R and B in line-sequence. With a Bayer array, G signals are present in all phases, both horizontal and vertical, but R and B are in line-sequence and accordingly signals corresponding to these only exist every other line in the horizontal or vertical direction. That is to say, the sampling frequency of R and B is half of the sampling frequency of G, so the limit of reproducible image signal frequency is also ½ for R and B as compared with G.
Accordingly, in the event that high-frequency components are present in the image signals, a phenomenon can occur wherein the G component of the high-frequency components can be correctly reconstructed but deterioration in amplitude and aliasing occur in the R component and B component, observed as offset in color balance in the output image. This phenomenon is known as false color (color moiré). Moreover, while the sampling frequencies of R and B are the same, the sampling phases differ. Accordingly, the color offset due to aliasing differ between R and G on the image, even further intensifying the false color phenomenon of R and B.
Various attempts have been made in conventional mosaic processing to reduce this false color phenomenon which deteriorates the image quality of output images. For example, interpolation processing has been proposed wherein the correlation as to neighbor pixels is detected for each pixel, and the interpolation method is switched corresponding to the correlation (e.g., see Japanese Unexamined Patent Application Publication No. 10-150668 (FIG. 1)). With this related art, following calculation of the degree of correlation in the vertical, horizontal, and diagonal directions, the direction in which the degree of correlation is the greatest is interpolated, thereby suppressing deterioration in resolution in the orthogonal direction thereof, and reducing occurrence of false color.
Also, with the above-described related art. The interpolation direction can be determined in the wrong direction near the Nyquist frequency, and accordingly a technique has been proposed wherein color saturation is calculated from color difference signals obtained by performing interpolation processing in either the horizontal or vertical direction by correlation determination, comparison is made with color saturation calculated from color difference signals obtained by performing interpolation processing without performing direction selection, and the color difference signals with the smaller color saturation (i.e., the one which can be though to have less occurrence of false color) is selected (e.g., see Japanese Unexamined Patent Application Publication No. 2002-300590 (FIG. 1)). Accordingly, occurrence of false color is reduced by performing interpolation without performing direction selection in a case of erroneous judgment in the correlation determination near the Nyquist frequency.